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Abstract. Understanding of short-term synaptic depression (STSD) and other forms of synaptic 
plasticity is a topical problem in neuroscience. Here we study the role of STSD in the formation of 
complex patterns of brain rhythms. We use a cortical circuit model of neural networks composed of 
irregular spiking excitatory and inhibitory neurons having type 1 and 2 excitability and stochastic 
dynamics. In the model, neurons form a sparsely connected network and their spontaneous activity 
is driven by random spikes representing synaptic noise. Using simulations and analytical calcula- 
tions, we found that if the STSD is absent, the neural network shows either asynchronous behavior 
or regular network oscillations depending on the noise level. In networks with STSD, changing pa- 
rameters of synaptic plasticity and the noise level, we observed transitions to complex patters of 
collective activity: mixed-mode and spindle oscillations, bursts of collective activity, and chaotic 
behaviour. Interestingly, these patterns are stable in a certain range of the parameters and separated 
by critical boundaries. Thus, the parameters of synaptic plasticity can play a role of control parame- 
ters or switchers between different network states. However, changes of the parameters caused by a 
disease may lead to dramatic impairment of ongoing neural activity. We analyze the chaotic neural 
activity by use of the 0-1 test for chaos (Gottwald, G. & Melbourne, I., 2004) and show that it has a 
collective nature. 
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INTRODUCTION 

Short-term synaptic depression (STSD) is an important form of short-term plasticity 
that provides a dynamic gain-control mechanism enhancing the sensitivity of cortical 
neurons to afferent firing patterns and expanding the range of possible coding strategies 
for cortical neurons [1, 2]. Recent experimental studies and phenomenological model 
of STSD (so-called Tsodyks-Markram (TM) model, [3]) have reported that synaptic 
transmitters are bombarded with the probability of release determined by the history 
of the synapses and by the arrival of the presynaptic spike (spike-timing dependent 
plasticity, STDP). Hence, synaptic efficacy is changed and adapted according to the 
dynamics of presynaptic and postsynaptic neurons. In turn, changes in synaptic efficacy 
influence activity of neurons. Thus, interplay between STSD and neuronal activity is 
an underlying mechanism that influences collective dynamics of neural network, in 
particular, brain rhythms. At the present time, understanding of this influence is elusive. 

The brain is always surrounded by noise and also noisy itself. Intuitively, noise 
is damaging. However, in neural networks, noise can play a positive role supporting 
oscillations and synchrony [4, 5]. Ergo, stochastic processe is also one of the important 



ingredients of neural network. 

In the present work, we study the role of STSD in the formation of complex patterns of 
neuronal activities. We use a cortical circuit model of stochastic neural networks having 
complex architecture and two types of neurons [6]. For describing STSD, the TM model 
is applied to excitatory-excitatory synapses. By simulations, we observed that STSD 
yields a rich reportoire of neuronal activities, such as mixed-mode oscillations (MMOs), 
spindles, and chaotic behaviour. The parameters of synaptic plasticity can play a role of 
control parameters or switchers between different network states. We analyze the chaotic 
neural activity by use of the 0-1 test for chaos [7, 8] and show that it has a collective 
nature. 



CORTICAL NETWORK MODEL 
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FIGURE 1. Input-output function: firing rates (or spike frequencies) of excitatory and inhibitory neu- 
rons versus input V. These firing rates follow sigmoid functions mimicking experimental reports. Square 
points and solid lines are for excitatory neurons. Circle points and dashed lines are for inhibitory neurons. 

Stochastic neurons. Let us consider two types of neurons: excitatory and inhibitory 
neurons. The total number of neuron is N. The fractions of excitatory and inhibitory 
neurons are g e and gi = 1 — g e , respectively. Neurons are linked by directed edges and 
form a network with an adjacency matrix a nm , where m,n = 1,2, ...,N. An element a nm is 
equal to 1 if there is an edge directed from a neuron m to a neuron n; otherwise, a nm = 0. 
If a neuron n is connected to a presynaptic neuron m, the synaptic efficacy in neuron 
n is J nm (t). We assume that all synapses of excitatory neurons are excitatory and all 
synapses of inhibitory neurons are inhibitory: this is the so-called Dale's principle [9]. 
Each neuron can be in either an active or inactive state. We define s n (t) = 1 if neuron 
n is active at moment t, and s n (t) = if this neuron is inactive. Active neurons fire 
random trains of spikes with a Poisson inter-spike intervals distribution [4, 10]. We take 
no account of phase correlation between trains of random spikes generated by different 
neurons. 




Network structure. We consider network structure is of a sparse random uncorre- 
cted directed network, so-called directed Erdos Renyi graph. These networks are small 
worlds and can have an arbitrary degree distribution. They are often considered as a 
good approximation to real networks [11, 12, 13]. The advantage of these networks is 
that they can be studied analytically by use of mean-freld theory and easily modeled for 
simulation. 

Noise. Each neuron is stimulated by Gaussian external noise with a mean rate 
in the range from to 20 kHz. This is the only source of the noise in our model. 
We assume that the noise is regarded as the signal from another brain area and also 
describes synaptic bombardment. In simulations, we consider a Gaussian function of 
the probability distribution that a neuron obtains t, (t) random spikes via extrinsic stimuli 
each time interval x (so-called integration time). 

Input-output. Neurons demonstrate various types of spiking behaviour in response 
to a stimulus at firing threshold [14, 15]. In particular, there are two types of threshold 
behaviour for periodcally firing neurons, which show continuous (type 1) or discontinu- 
ous (type 2) input-output curves [16, 17]. Excitatory neurons (pyramidal cells) represent 
type 1, whereas inhibitory neurons (interneurons) show type 2 behaviour. In figure 1, 
we mimic the mean firing rates (in the present paper, we refer to as just "firing rates ") 
described by sigmoid input-out functions of neurons reflecting the experimental reports. 
At every integration time x, all neurons are updated in parallel, in which postsynaptic 
neuron receives and integrates spikes from active presynaptic excitatory and inhibitory 
neurons and also Gaussian external input. 

In a nutshell, the total number of input per neuron is: 

N 
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where k n (t) is the number of spikes that arrive from presynaptic neuron n and < t, > is 
the mean number of random spikes during the time interval [t — x, t] . Correspondingly, 
probabiliy distribution to get k n spikes from an active excitatory or inhibitory presynaptic 
neuron, n obeys a Poisson distribution. The inverse function of a mean firing rate 
indicates an average value of inter-spike interval times. Here we consider the case of 
xf < 1, that is, a postsynaptic neuron recieves only one spike (k n = 1) or none (k n = 0) 
from an active presynaptic neuron during the integration time x, thereby the parameter 
xf has a meaning of the probability that a postsynaptic neuron receives a spike from an 
active presynaptic neuron during time x. 

Activation- deactivation. If the total input V m {t) at an inactive excitatory or inhibitory 
neuron m at time t is at least a certain threshold £l a (i.e. V m (t) > Q. a ), then the neuron 
m fires with a activation rate, \x a . And the active excitatory (inhibitory) neuron m is 
inactivated at a rate \l a if V m (t) < Q. a , provided that a = e for excitatory and a = i for 
inhibitory neurons, respectively. The activation rate, \l a is the reciprocal function of 
the mean first spike latency, At s i. Thus, activation of neurons also follows a stochastic 
process with a characteristic time (t r = At s i). For simplicity, we assume that activation 
and deactivation rate \i a are the same and do not depend on the input. The relation 
between excitatory and inhibitory activation rates is /i ; = a\l e 



Short-term synaptic depression. The model of short-term synaptic depression is 
based on depletion of a pool of vesicles as a synaptic resource which is analogous to 
the probability of release in the phenomenological model described in (markram et al. 
1998 [3]). It reflects the assumption that a releasable pool is determined not only by the 
history of the synapse but also by the arrival of the new presynaptic spike (STDP). As a 
dynamical synapses, we only take synapses between excitatory neurons into account in 
the present work. Other synaptic efficacies are constant. J e (t) = Joy(t), Jo is the absolute 
strength and y(t) is a scaling factor which is the fraction of releasable synaptic resources. 
The rate equation for evolution of y is described by the following: 

^ = — -P d y8(t-t s ), (2) 

at Zr 

where the control parameter P d and is the degree of depression (with < Pj < 1) and 
the recovery time from synaptic depression, respectively, and t s is the arrival timing of 
the presynaptic spikes, 

The parameters can take any real values observed in experiments. For the sake of 
convenience, we use dimensionless units for all parameters. In simulations, the model 
parameters are set as followings: the network size N = 10 3 - 10 5 ; the mean degree 
10 3 ; the fraction of inhibitory neurons gi = 0.25; the all excitatory efficacies J e = 1 
and all inhibitory efficacies 7, = —3; the threshold Q. e = 25, Q, = 27 for excitatory and 
inhibitory neurons, respectively; the maximum mean firing rate of excitatory neurons 
j-max _ ^ anc | Q f inhibitory neurons/J" ax = 3; the activation rate of all excitatory neurons 
f± e = \; the deviation of the Gaussian random noise o = 10; the ratio between first spike 
latency of excitatory and inhibitory neurons /!,/ \x e = a= [0, 1] . the fraction of available 
synaptic resources P d = [0, 1]. These parameters agree with measurements in cortex. 



RESULTS 

In order to describe the dynamics of cortical neural networks, let us define p a (t) for 
excitatory, a = e, and inhibitory, a = i, neurons, 

Pa{t)^Pn\t)/iSaNl (3) 
n 

where n is index of neuron and the sum is over neurons of type a, g a is their fraction. 
p e (t) and pi(t) are referred to as "activities"of the excitatory and inhibitory populations. 
In other words, p a (t) is the respective probabilities that a randomly chosen excitatory 
(a = e) or inhibitory (a = i) neuron is active at time t. This activity is analogous to 
EEG measurements of brain waves. To clarify the effects of depressing synapses, first 
we study a cortical network model in the absence of synaptic plasticity. This model has 
a complex phase diagram with self-organized top-down states, hybrid phase transitions, 
hysteresis phenomena, and a rich array of behaviours including decaying and stable 
oscillations, stochastic resonance, and neural avalanches. It is also shown that global 
oscillations and stochastic resonance are intrinsic properties of this nonlinear dynamical 
system (also refer [6]). By simulation, we found the phase diagram with three different 



regions on the a— < £, > plane as shown in Fig. 2. In neural networks without synaptic 
plasticity, neuronal activities, p a show different collective behaviours - sustained I 
or suppressed states II, and global oscillations III depending on the parameter set, 
{a, < ^ >}. The theory and simulations are in very good agreement at N = 100000. To 
investigate the effect of STSD on neural networks in the formation of complex patterns, 
we choose representative points on the phase II, III, and near the phase boundary 
between II and III (see the points A, B, C in Fig. 2). The given points, A, B, C have 
the fixed average external stimulus, < ^ >= 15 and different control parameter a = 0.2, 
0.6, 0.8 for the oscillations, mixed states (II and III phases), and sustained active states, 
respectively. 
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FIGURE 2. Phase diagram, a-< ^ > plane: I. suppressed states, II. sustained active states, III. regular 
oscillations. Points A, B, C show the parameters chosen for STSD; A. a = 0.8 and < ^ >= 15, B. a = 0.6 
and < 4 >= 15, C. a = 0.2 and < % >= 15. 

By STSD, neuronal activities turn out various brain rhythms which are experimentally 
observed in vitro and in vivo mammalian brain. Through the given points A, B, C 
of the figure 2, in certain rages of synaptic parameters, we observed the new phases 
of neuronal activities which show chaotic behaviours. Emergence of chaos is very 
intriguing phenomenon which is measured in EEG brain rhythms [18, 19]. 

Regular oscillations of neuronal activity reveal amazing robustness against noise and 
depression of synapses in a broad range (1 — 30 Hz) of parameters. However, we found 
that sufficiently strong synaptic bombardments (above 8 Hz) stimulate synchronization 
between neurons and global neural oscillations when the synaptic depression is large 
enough. With parameters a = 0.2 and < B, >= 15 (point C in the plane of a— < t, >), 
as depression parameter P c { and/or recovery time Xr are increased, new activity occurs to 
a certain extent in the intermediate region between regular oscillations and suppressed 
states. The new activity has complex patterns that arise in neural networks, in which 
oscillations with different amplitudes are interspersed, that is, the neuronal activity 
switches between small amplitude oscillations and large amplitude oscillations. This 
complex rhythm is reffered to as mixed- mode oscillations (MMOs). Figure 3 shows a 
sample snapshot of MMOs in time, where synaptic depression parameters are = 0.01, 
i R = 51. 
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FIGURE 3. Mixed-mode oscillations (or period-doubling oscillations): neuronal activity versus time. 
Parameters are a — 0.2. < % >= 15, P ( / = 0.01, = 51. Solid lines are for excitatory neurons, whereas 
dahsed-dotted lines are for inhibitory neurons. 



In the case of mixed state without plasticity (parameter a = 0.6, < £ >= 15, point B 
in Fig. 2), if neural network goes through STSD in certain range of parameters {Pj, Tr}, 
new phase of a so-called spindle-like oscillations appears. Spindles are particularly 
generated during early stages of sleep, we observed these sleep spindles in our cortical 
model. For example, with parameters Pj = 0.008 and Xr = 9, we found 4 — 5 Hz spindle- 
like oscillations about 24 spindles/min (see fig. 4-(b)). 
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FIGURE 4. the 0-1 test for chaos: neuronal activity vs. time, (a) K c = 0.166197 for regular oscillations 
(a = 0.2, < % >= 15, P d = 0.002, x R = 4), (b) K c = 0.764383 for spindle oscillations (a = 0.6, < % >= 
15, P rf = 0.008, T/f = 9), (c) K c = 0.971950 for chaotic oscillations (a = 0.8, < E, >= 15, P rf = 0.004, 
Tr = 2.5). Square points and solid lines (gray color online) represent original signal of excitatory neurons 
while circle points and dashed lines (blue color online) show smoothing points (activities at every 5 
integration times, 5t). 



Lastly, when STSD is applied to initially sustained-active state (point A in Fig. 2, 
a = 0.8, < t, >= 15), the neural network discloses another new phase in a certain range 
of parameters {P d , Tr}. The new phase exhibits chaotic activities as shown in Fig. 4-(c). 
Typically MMOs and spindle oscillations have been also regarded as a chaotic behavour. 
Here we make a narrower definition of chaotic behaviour. We primarily refer to chaotic 
activity as complex pattern that is lack of phase and frequency synchronization. 

To check whether the observed activity in our model is chaotic or not, we use recent 
method, namely the 0-1 test for chaos introduced by Gottwald & Melbourne [7, 8]. As 
K c ~ 1, the rhythm is wholly chaotic. For a sample chaotic signal with the parameter 
set {a = 0.8, < t, >= 15, P d = 0.004, x R = 2.5}, we obtained K c = 0.971950 (Fig. 4- 
(c)). thus, it is confirmed that our observed signals are chaotic. Whereas, we acquired 
K c = 0.764383 from the data in the regime of spindles with the parameter set {a — 
0.6, < % >= 15, P d = 0.008, Tr = 9} (Fig. 4-(b)) and as would be expected, K c ~ in 
the regular oscillation regime (Fig. 4-(a)). 

CONCLUSION 

By studying a cortical circuit model, we understand the role of STSD in the formation of 
complex patterns of brain rhythms. Neuron is regarded as a generator of random spikes 
with a Poisson inter-spike intervals distribution and has type 1 (excitatory neuron )or 
type 2 (inhibitory neuron). Firing rate of excitatory (inhibitory) neuron is described by 
sigmoid function of input-out, And synaptic input from extrinsic afferents is modeled as 
an independent Gaussina process. Noise is a driving force and plays a crucial role in the 
dynamics of neural network. We use TM model [3] for STSD of excitatory-excitatory 
dynamical synapses. By performing intensive simulations, we show that if the STSD is 
absent, the neural network exhibits either asynchronous behavior or regular network 
oscillations depending on the noise level (in agreement with analytical calculations 
and numerical analysis but with Heavy-side step function of firing rates). Significantly, 
we also unravel the role of STSD in the present noise whereby depressing excitatory 
synapses not only suppress neuronal activities but also on the contrary lead to a rich 
pattern of neural oscillations such as, mixed-mode oscillations, spindles, and chaotic 
activity. We analyze the chaotic neural activity by use of the 0-1 test for chaos [7, 8] 
and show that it has a collective nature. These patterns are stable in a certain range of 
the parameters and separated by critical boundaries. Thus, the parameters of synaptic 
plasticity can play a role of control parameters or switchers between different network 
states. However, changes of the parameters caused by a disease may lead to dramatic 
dysfunction of ongoing neuronal activity. 
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